Computer-automated system and method of assessing the orientation, awareness and responses of a person with reduced capacity

ABSTRACT

A system and method for scheduling, and reminding people receiving care (PRC) about upcoming events uses a remote computer-based interface that allows a Caregiver at a remote location to schedule the events for the PRC. The PRC has in their residence (or other location) a device that prompts the PRC, by an audio and/or video prompt, so as to “nudge” the PRC, asking whether they intend to participate in a pre-scheduled event. A man-machine interface, such as a two input keypad containing one button labeled “yes” and another button labeled “no”, the remote Caregiver is able to monitor the responses of the electronic “nudges” offered by the system to the PRC. In turn, the device used by the remote Caregiver accumulates the responses and creates a history of the level of activity the person has indicated in which they wish to participate.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of and is based upon and claims thebenefit of priority under 35 U.S.C. §120 for U.S. Ser. No. 10/910,569,filed Aug. 4, 2004, the entire contents of each which is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to computer-automated interactive systemsand methods of assessing the orientation, awareness and responses of aperson with reduced capacity (hereinafter “PRC”). More particularly, thesystem and method is directed to addressing scheduling activities andspecialized instructions for the PRC, monitoring their interactiveresponses to invitations to participate in the activities, andsubsequently, determining whether their responses are corroborated bythe actual detected activities of the PRC. The present invention furtherprovides a computer-automated system and method of performing automatedactions in accord with the responses and actual detected activities.

2. Discussion of the Background Art

Though the present invention may be applied to the benefit of any PRC,for ease of understanding, some of its applications are hereinafterexplained with respect to elderly persons. Moreover, a PRC should not beconstrued narrowly and includes anyone (blind, aged, hearing impaired,inexperienced younger people, non-native English speaking people,crippled, mentally challenged, etc.) who would benefit from aninteractive system and method for providing reminders. In the specialcase where the PRC is an elderly person, because the elderly are moreapt to incur a wide range of reduced capacities, such as cognitive,sensory, physical, and emotional capacity, they provide an excellentcontext within which to explain the present invention.

The United State's population of elderly is expected to double to 70million by 2030. Much of that rise is due to the 76 million baby-boomersthat will begin turning 65 years-of-age in 2011. This shift indemographics in the Untied States (for example) will give rise to veryserious emotional and financial stress to a greater percentage of peoplewho are the younger members, responsible for caring for their moreelderly relatives. The cost of care for elderly people can beprohibitive, even for middle-class wage earners. An important aspect ofa well-balanced financial plan for a modern family is to encourage allfamily members to take care of themselves and be self-sufficient for aslong as possible so as to avoid being a drain on the financial resourcesof the rest of the family, and/or having to apply for governmentassistance. In addition to providing an enormous market for technology,the elderly (and of course most PRCs, and their families) stand togreatly benefit from technological advances for at least the reasonsstated below.

Two areas in which the elderly are often in greater need are activityand supervision. Regarding the former, the elderly are more susceptibleto inactivity caused by general depression (e.g., loneliness andmalaise), physical discomfort (e.g., arthritis and fatigue), and lack ofopportunities (e.g., lack of transportation or forgetfulness of upcomingevents). The later cause of inactivity frequently results from aCaregiver's failure to apprise an elderly person of available activitiesor failure to realize that activities are currently unavailable to thatperson. Regardless of the reason, the present inventors recognize thereis a need to notify, or simply remind, the elderly of availableactivities and to monitor that they are generally participating in thesame.

Furthermore, the elderly are also more susceptible to severe and suddendeclines in orientation, awareness and responses, such as being due tosevere depression and its symptoms (e.g., failure to eat); and severephysical ailment (e.g., strokes and heart attacks). Unlike theabove-noted problems, which should not be overshadowed, severedepression and physical ailment must be immediately addressed to preventpermanent or persistent disability or death. Accordingly, there is aneed to immediately notify a Caregiver of inactivity, suggesting suchdepression or physical ailment.

U.S. Pat. No. 6,611,206, issued to Eshelman et al., is an example of aconventional technique for monitoring independent persons requiringoccasional assistance. The system relies on the presence of a largenumber of different types of sensors (including video and audio),coupled with artificial intelligence programs, to monitor the activitiesof a person needing occasional assistance. In the several examplesdescribed, such as Example 1, cameras are aimed at a child sleeping in acrib, and a microphone is placed in a position to pick up sounds. Acontroller is programmed to recognize the normal look of the baby's faceand recognize the sound of crying and produce a signal that cryingoccurs. The controller is also configured to analyze the facialexpressions of the infant so as to determine whether the infant iscrying, playing, in distress, or expressing other moods that may beremotely observed by a caretaker. In a second example, an elderly personlives alone and is observable via multiple video cameras and audiomicrophones located throughout the house. These sensors are able torecognize the activity level of the person and when used with acontroller, are able to assess their particular mood. For example, usingartificial intelligence, the system may be trained to recognizefrustration in detecting the elderly person's utterance of periodicclipped speech using words indicative of a negative mood.

The present inventors have recognized that a limitation withconventional approaches such as that described in U.S. Pat. No.6,611,206 is that elderly people merely need occasional assistance, andhave no interest in losing their dignity by having all of theiractivities monitored by “Big Brother”. A common limitation with elderlypeople is forgetfulness. A system such as that described in U.S. Pat.No. 6,611,206 would be overly intrusive simply for the purpose ofreminding a person about a particular planned activity.

There are conventional scheduling software applications, such asMICROSOFT OUTLOOK, that permit users to schedule an event, to generatemessages reminding themselves or others that the event is impending, togenerate invitations requesting a response as to whether the inviteewill attend, and to monitor the responses of the invitees. With theadvent of digital networks, e.g., the Internet, scheduling applicationshave further permitted users to perform these functions via remotedevices; and to remotely receive and respond to the reminder messagesand/or invitations. Thus, presumably, some Caregivers might try to use aconventional scheduling application to schedule an upcoming event,notify an elderly person of the event, and determine whether that personintends to attend the event (e.g., via R.S.V.P.). It bears mention thatthe success of such an endeavor may indicate the elderly person is not aPRC in need of such services.

FIG. 9 is a flow chart showing a conventional process for monitoring theactivities of a PRC. The process begins in step S101, where specificsensor tags are attached to objects whose movement indicates dailyactivities of the PRC. For example, the sensor tags would include atoothbrush, coffee mug, and other objects that typically would be usedby a PRC who is functioning self-sufficiently with regard to thoseparticular objects. Subsequently the process proceeds to step S102,where the sensor tags are also attached to particular devices associatedwith taking the PRC's medications. As an example, the sensors may beattached to the medicine cabinet doors, pill boxes, or pill dispensers.Subsequently the process proceeds to attaching sensor tags to theelderly person's body, perhaps in the form of a bracelet, and attachingsmart sensors to where the measurement of the respective sensorsindicate specific psychological conditions. As one example, the sensorsmay be placed in the person's shoes in order to indicate theregular/irregular gait of that particular person.

The process then proceeds to step S105 where the movement of therespective sensors are analyzed to determine the level of activity ofthat person. The process then proceeds to assess the data received bythe sensors arranged in step S104 to determine the specificpsychological condition of the PRC. Based on the data received in theabove steps, the process proceeds to step S107, where the Caregiver isnotified of inadequate or irregular activity where indicated by theabove process steps.

Further, many operating systems, add-on software applications, andspecialty devices make the user interfaces of such schedulingapplications more accessible to PRCs. As a result, presumably, somecomputer-literate PRCs might use scheduling applications without theon-site assistance of another person, regardless of the nature of theirimpediment.

Further, sophisticated systems are currently available that monitor theactivity of a PRC at arms length. For instance, sensors are currentlyemployed to detect whether a PRC has awakened and used a bathroomfacility. Intel Corporation is reportedly testing a system of sensorsand devices for the elderly that issues medication reminders anddetermines an elderly person's level of activity. General Electric issimilarly reported to be testing a system that detects abnormal behaviorof an elderly person and accordingly transmits an automated alert ofthat behavior to a Caregiver.

FIG. 11 is a block diagram of a web-based system for sending photographsto be displayed on a photoviewer 1100 for people who are not computerliterate. CEIVA (see www.ceiva.com) sells a receiver 1100 that lookslike a picture frame, but is able to display digital photographs sent toif from a server 1105. Moreover, a family member uses their computer1103 to upload their photos to the server 1105 via a network (such asthe Internet and/or public switched telephone network). The receiver1100 at a predetermined time (e.g., 2 AM) dials into the server 1105 anddownloads photos to the receiver, which are then displayed as picturesin a photo frame. This system is quite convenient for people who are notcomputer literate, such as many PRCs, because once the system is set up,they need to do nothing, yet they are able to view updated photos sentfrom family members. Family members upload the photos by logging on tothe www.CEIVA.com website and responding to the prompts on the web pageto upload photos to the server 1105.

As the inventors of the present invention have recognized, none of thoseproducts provide a computer-automated system and method of assessing theorientation, awareness, and responses of PRCs by remotely schedulingactivities for them and then monitoring their interaction with thesystem, such as “yes”, “no” responses to invitations to participate inthose scheduled activities. It follows that none of the products comparea PRC's responses to such invitations against a sensed activity of thePRC; and none of the products permit a remote programming of automatedactions that are performed in accord with the PRC's responses to suchinvitations and/or in accord with whether the PRC's responses and sensedactivities are congruous with one another.

SUMMARY OF THE INVENTION

In view of at least the above-noted problems and deficiencies of thebackground art, an object of the present invention is to provide aremote scheduling and monitoring system that addresses theabove-identified and other limitations of conventional systems andmethods. The system and method of the present invention uses a remotecomputer-based interface that allows a Caregiver at a remote location toschedule events for the PRC. Alternatively, the scheduling may be doneon the device that the PRC will actually use. In either case, the PRChas in their residence (or other location) a device that either audio orvideo prompts the PRC to “nudge” them, asking whether they intend toparticipate in a pre-scheduled event. Using a simple man-machineinterface, such as a two input keypad containing one button labeled“yes” and another button labeled “no”, the remote Caregiver is able tomonitor the responses of the electronic “nudges” offered by the systemto the PRC. In turn, the device used by the remote Caregiver is able toaccumulate the responses and create a history of the level of activitythe person has indicated they wish to participate in. This data may thenbe corroborated with reports from third parties (and/or sensors)indicating whether the third party has actually participated in thoseevents.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the present invention and many of theattendant advantages thereof will be readily obtained as the samebecomes better understood by reference to the following detaileddescriptions of the present invention, and when considered in connectionwith the accompanying drawings, wherein:

FIG. 1 is a system block diagram of an embodiment of the presentinvention;

FIG. 2 is a process flow diagram showing a general overview of stepsprovided by an embodiment of the present invention;

FIG. 3 is a block diagram of a local interaction device used by the PRC;

FIG. 4 is a flow diagram of a process for scheduling an activity for aPRC;

FIG. 5 is a flow diagram of a process for inviting the PRC toparticipate in a scheduled activity;

FIG. 6 is a flow diagram of a process for performing automated actionsin accord with a response of the PRC;

FIG. 7 is a flow diagram of a process for comparing data received fromthe local interaction device, and comparing the same againstpredetermined thresholds for determining a status of the PRC;

FIG. 8 is block diagram of a web-based embodiment of the presentinvention;

FIG. 9 is a block diagram of a computer that may be used to implementthe processing and communication steps and operations of the presentinvention;

FIG. 10 is a flow diagram of a process for monitoring a subjectaccording to the Background Art; and

FIG. 11 is a block diagram of a conventional system for downloadingphotos to an individual for viewing on an electronic viewer, which isnot a personal computer.

DETAILED DESCRIPTION OF THE INVENTION

Referring now to the drawings, wherein like reference numerals designateidentical or corresponding parts or steps, FIG. 1 illustrates a systemblock diagram of a PRC scheduling and interface system according to thepresent invention. A local interaction device 1, commonly placed in theresidence of the PRC, is connected by way of a communication link 9 byway of a router or switch 7 (optional) as part of a network 5 to aremote interface 3. Moreover, this connection between the localinteraction device 1 and the remote interface 3 may be performed by anyone of a number of ways, including direct physical connection (e.g.Ethernet, dialup network, Internet connection, wireless link, or hybridcombinations). The network 5 also connects to various third parties,such as equipment (like the remote interface 3) used by a therapist 11,a pharmacist 13, and social director 15, as will be discussed.

The local interaction device 1 is the device located near the PRC and isused by the PRC to respond to notifications or alarms scheduled by aCaregiver. As an example, the remote interface 3 is used by a Caregiverto set a particular event for announcement to the PRC by way of thelocal interface device 1. This announcement may be, as an example, avideo announcement displayed on video device 25 (e.g. a TV, a computermonitor, or a local display on the local interface device). The alarmmay also be an audible alarm or a combination of an audible alarm and avideo announcement, where the audible alarm is produced by an audiointerface 23 that includes a speaker(s). A timer/scheduler 21 recognizeswhen the time has arrived to dispatch a command for making the scheduledannouncement to the PRC.

In response to the command, the scheduler 21 sends a signal over bus 17to the video device 25 and/or audio device 23 to announce to the PRCthat the event is coming soon, and inquires whether the PRC plans toattend. The PRC can then reset the alarm by interacting with the localinterface 9, as will be discussed in more detail with regard to FIG. 3.Typically, the local interface 19 provides two responses (affirmative,or negative) by the PRC that will be recognized by the local interactiondevice 1. One response is made by the PRC pressing a button labeled“yes”, indicating that the PRC would like to participate in the event.The local interface 19 also includes a second button labeled “no”, whichif pressed by the PRC would indicate that the PRC does not have anintention to attend the event.

As will be appreciated from the discussion with regard to FIG. 3, thelocal interface 19 need not be limited to the use of buttons, but mayalso use other man-machine interfaces configured to detect the PRC'spreference. As an example, the local interaction device 1 may receive anaudio indication from the PRC (such as a spoken work “yes”), indicatingthe PRC's intentions with regard to the scheduled event. Likewise, thevideo device 25 may take a short video clip of the individual when theindividual steps within a certain proximity (e.g. 10 feet) of thedevice, as sensed by a motion detector or tactile sensor (neither isshown). The PRC may indicate his or her preference by either shaking hisor her head, indicating “no”, or nodding his or her head, indicating“yes”. These physical actions are then recorded in a video clip (orfile) that is saved to memory and subsequently transmitted to the remoteinterface 3 at a predetermined time (such as when requested by anoperator of the remote interface 3). The video file (or more generally“interaction data”) is then sent by way of an input/output (I/O) device27 and the communication link 9 (perhaps including the network 5 androuter 7) to the I/O device 33 in the remote interface 3. This videofile is provided to the monitor device 31, which collects the data andcompares the same against a prestored threshold.

As an example, when the PRC provides a tactile response by way of thelocal interface 19, indicating his or her preference. The monitor 31ultimately receives the corresponding interaction data and stores thesame for comparison against the predetermined threshold. In the case ofthe audio signal provided from the PRC, the monitor 31 uses a voicedetector to determine whether the individual said yes or no (as anexample) indicating his or her preference. Similarly, with the videosignal is left by the PRC, the monitor 31 performs image recognitionprocessing to determine whether the individual is shaking his or herhead sideways (indicating a negative response), or nodding up and down(indicating an affirmative response). On the other hand, the system mayalso save the audio and video replies for audible and visual observationby the Caregiver.

In an alternative embodiment a separate monitor 31 is also included inthe local interface device 1. This way, a person visiting the PRC mayretrieve the data collected at the local interface device 1.

Regarding the scheduler 29 in the remote interface 3, the Caregiver usesthe scheduler 29 to input (in text format as an example) a reminder fora certain activity for the PRC. The reminder may indicate, as anexample, that there is a bridge game scheduled for a predetermined timeduring that day, and at predefined intervals (hourly intervals prior tothe event) input from the scheduler 29 which is downloaded by way of theI/O 33 and communication link 9, and is stored in the timer/scheduler 21of the local interaction device 1. In this way, the Caregiver may set inadvance particular events for the PRC to be made aware of, and requiringthat the PRC provide some interaction with the local interaction device1, indicating his or her preference for attending the event or not. Thescheduler 3 is implemented in software and provides a graphical userinterface, prompting the user for data to be entered. A same scheduler29 may be included in the local interaction device 1.

Similarly, aside from Caregivers, a therapist 11 may have a similarsystem to the remote interface 3 (perhaps without the monitor, but notnecessarily) so that the therapist can schedule particular events andprovide reminders to the PRC. For example, perhaps the therapist hasarranged for a series of walks for the PRC during the day. The therapistcan set the schedule for the PRC and provide reminders for the PRC tocomply with the therapists' request. The therapist may then monitor theresponses of the PRC to see if the PRC is properly adhering to thescheduled exercises.

Similarly, a pharmacist 13 (or even a physician, not shown) may use theremote interface 3 to input reminder data for the PRC to take certainmedications at certain times during the day and request confirmation (ormerely monitor the responses) that the PRC has actually taken themedication. Likewise, a social director 15 may schedule socialactivities for the PRC and request confirmation that the PRC intends toattend the scheduled events.

For the most part, the description of the present invention has beenmade in the context of the local interaction device generating areminder prior to an even occurring. However, the present invention isalso useful to generate reminders after an event has been detected. Forexample, when a signal is generated by a sensor (e.g., a magnetic reedswitch) attached to a door, detecting that the door was opened, thesignal is provided to the local interaction device 1 via the I/O 27. Thesignal triggers a reminder (audio, video or both) for the PRC to dosomething. Some memory-challenged people often forget where they haveplaced their purse, or their keys. In this situation, theTimer/Scheduler 21 is programmed to generate an audio announcement,reminding the PRC to place their purse or keys in a predeterminedlocation, such as “put the keys in the bowl.”

In this embodiment, MICROSOFT WINDOWS 2000 and MICROSOFT OUTLOOK may beinstalled in a typical fashion on the local interaction device 1.OUTLOOK's built-in calendar may be used as the engine managing thescheduling of activities, but OUTLOOK's user-interface is preferablyreplaced with a customized set of user screens programmed via a standardprogramming language (e.g., MICROSOFT VISUAL BASIC).

A system-level description of steps performed according to the presentinvention is discussed with reference to FIG. 2. The illustrated stepsare subsequently discussed in greater detail with reference to the otherfigures. More particularly, steps S210, S220, S230, S240, S250, and S260are further described with reference to later Figures.

FIG. 2 shows that the Caregiver first remotely accesses the localinteraction device 1 (S210), using for example GOTOMYPC software, toschedule an activity for the PRC (S220). Alternatively, the Caregivermay directly access the local interaction device 1 onsite by way of thetimer/scheduler 21, or separate scheduler 29, if installed in the localinteraction device 1. The Caregiver enters first, second, and third setsof information to schedule activity (S220), respectively. The first setmay include schedule information such as the activity name, date, time,duration, location, and activity reoccurrence. The second set mayinclude activity information such as the activity description, arrangedtransportation, attire, other indicating they will attend, and otherinformation. The third set may include action information such as theautomated actions corresponding to the PRC's response to a respectiveinvitation (e.g., email daughter@work.com, if there is no response tothe invitation repeated a predetermined number, such as 5 times).

An optional feature of the system is the inclusion of a graduatedinteraction mechanism included in the timer/scheduler 21 included aspart of the local interaction device 1. The graduated interactionmechanism allows a computer savvy PRC to set his or her own scheduledreminders when at a highest grade-level of interaction, but later limitsthe amount of programmability by the PRC to a lowest grade-level whenthe PRC's abilities degrade. The number of gradations (e.g., grades 1through 5) is not critical, as long as there is an association betweengrade level and degree of autonomy the PRC has over the scheduling andmonitoring functions of the system. As an example, a newly retired orvery moderately impaired individual who has computer skill and Internetcapabilities might be highly involved in determining what activities tobe involved in, and input such to the system. As the PRC's abilities(either mental or physical) decrease, the timer/scheduler 21 is adjusted(manually, or remotely by the caregiver or PRC) to become morerestrictive (e.g., decrease from a grade level of 1 to 3, on a scale of1 to 5, for example), to perhaps allow the PRC to provide yes/noresponses, and schedule social activities, but not medicinal ortherapeutic, as an example. Later, as the PRC's abilities degradefurther, the grade of the timer/scheduler 21 is reduced to the lowestgrade, 5, where the PRC is allowed only to enter “yes” or “no”responses.

After the activity is scheduled (S220), the local interaction device 1notifies the PRC of a generated invitation (S230) by any number ofmeasures (e.g., produces a particular sound, flashes the householdlights, or transmits to a pager worn by the PRC), including simplydisplaying the invitation on the screen of the local interaction device1. Upon determining that the PRC is ready for the invitation (anoptional step, in which the PRC provides feedback that the PRC wants toreceive the invitation), the device announces the invitation (S240) in aformat tailored for comprehension by the PRC (e.g., displaying largefont text and utilizing the aforementioned text-to-speech software, aswell as the other notice functions discussed above).

The PRC may respond to the invitation (S250) by indicating “YES”, “NO”,or providing no response. In the present embodiment, the PRC willprovide a tactile response “Yes” or “No” by activating the oversizedbuttons on the local interface 19. Alternatively, the PRC may alsorespond by requesting “MORE INFORMATION”. In either matter, eachresponse triggers the local interaction device 1 to perform an automatedaction (S260), which is tailored to the action information entered bythe Caregiver in step S220 and based on the PRC's response, sensedactivity, or both.

FIG. 3 is a block diagram of the local interaction device 1. The localinteraction device 1 may be a standalone device, having a processor andI/O for connection over a network (such as a modified version of theelectronic picture frame 1100 previously discussed with regard to FIG.11). Alternatively, and as shown, is a peripheral device detachablyattaches to a personal computer by way of a peripheral bus, such as aUSB bus, FIREWIRE interface, or the like. The local interaction device 1includes a microphone 41 (optional) for receiving audible input providedby the PRC. For example, the PRC may provide an audible response inreply to a scheduled event that is used to notify the PRC of theupcoming event. One or more speakers 43 are also included eitherembedded in the local interaction device 1 or attached thereto. Thespeakers provide audible announcements from the local interaction device1 for listening by the PRC. The local interaction device includes anoptional display 39 that can display text messages, or pictures,indicating the upcoming event. As an example, the picture may be adigital file (as will be discussed with regard to FIG. 8) that isdownloaded from the remote interface 3 (FIG. 1). Moreover, the display39 may be more suitable for some PRCs who would rather receive a visualdisplay, rather than an audio alarm.

The local interaction device 1 optionally includes a pan, tilt, zoomcamera 37 that may be connected to a motion detector. Moreover, when thePRC moves within a predetermined distance of the local interactiondevice 1, the PTZ camera 37 is actuated, and records the activities ofthat individual (nodding one's head or shaking one's head). The PTZfeature of the camera 37 may be used for automatic tracking of the PRCor remotely controlled (zoom, tilt or pan) by way of the remoteinterface 3.

The “yes” button 30 and “no” button 33 provide a tactile interface forthe PRC to indicate his or her intent to reply to the subject of thealarm. For example, if an audible alarm announces a planned bridge gamein one hour, the PRC if he or she chooses to go presses the “yes” button30. The data would then be sent to the remote interface 3 (and alsostored locally) for keeping track of the data. Optionally, a “moreinformation” button 35 may be used to obtain any additional informationavailable from the remote interface 3 or the local interaction device 1,should the “more information” button 35 be pressed by the PRC.

Although the tactile interface shown in FIG. 3 is in the form ofbuttons, other interfaces may be used as well, such as pressure sensors,capacitively coupled switches, or a touch screen interface.

The local interaction device 1 may also be portable and battery operatedso as to be taken with the PRC to the particular activity. As anexample, because the information is located in the local interactiondevice 1, and because the local interaction device 1 may be aself-contained portable unit (such included in or on a PDA, cell phone,hearing aide, watch, pendant, bi-focal frame, or even a larger devicethat is adapted to contain a two-way wireless, e.g., infrared or radiofrequency communication capability) the schedule events may alert thePRC while the PRC is out of his or her residence. For example, if thePRC is participating in an activity, the PRC may need additionalguidance regarding now to return home. For example, if a bus isscheduled to pick-up the PRC at a predetermined time from a museum, thenthis mobile version of the local interaction device 1 will sound analarm alerting the PRC prior to the arrival of the bus. Also, if the PRCbecomes apprehensive about how he or she will return to his or herresidence, the PRC is able to view the activity on the local interactiondevice 1, informing the PRC about the issue that he or she is concernedabout.

FIG. 4 is a block diagram showing an example of how to enter a scheduledactivity for the PRC via the remote interface 3. As shown, the Caregiverfirst accesses a graphical scheduling information screen, and isprompted to enter schedule information which includes the activity name,day, time, location, and reoccurrences of the activity (S211). In thisinstance, the Caregiver has entered Bridge Club, Jan. 1, 2004, 4 p.m.,Senior Center, and Bi-weekly, respectively, for the scheduleinformation. Since “Senior Center” and “Bi-weekly” have been previouslyentered, those entries were entered via a pull down menu option.

Some of the schedule information entries are stored as text strings,e.g., “Bridge Club” is stored as a string corresponding to [activityname], to be later used when announcing the contents of the invitation;and some of the schedule information is stored as commands for automatedactions of the device 1. For instance, the entry “biweekly”corresponding to [reoccurrences] prompts the device 1 to repeat theinvitation (see below) every two weeks on date and time provided.Optionally, the Caregiver may elect to set the time(s) before the eventthe announcement is made, the duration of the announcement, and thenumber of times the announcement is made.

In step S212 the Caregiver is prompted to enter information describingsome of the administrative aspects of the activity. For example, thetransportation means for how the PRC will be taken and received, as wellas the attire.

After entering the schedule information and activity information, theCaregiver accesses the customized action information screen, and entersa corresponding action (or no action) for each permitted response of thePRC (S213). In this instance, again, pull down options are available forfrequently used entries. Two such entries are <log> and <analyze> whichcollectively prompt the device to log the response and determine (e.g.,calculating) whether the PRC's recent level of inactivity has exceeded aprescribed level.

In this instance, the Caregiver has also entered <e-mail:van@seniorcenter.org> to instruct the device to send an e-mail to thesenior center confirming the PRC will attend the Bridge Club meeting andneed a ride thereto. In addition, the action entered for the “Yes”response also includes the command/s “<monitor: door sensor> (if true)<email: daughter@work.com>”, which prompts the device to email thedaughter if a particular door sensor is not activated in the next 60min. Such notification would indicate that the door was not openedduring the time period in which the PRC should left the home for theBridge Club meeting.

Likewise, the therapist or social director who may be in charge ofmonitoring the activity that the PRC participates in, may input intotheir remote interface, an indication that the PRC has or has notattended. By this corroborating information, the Caregiver receivesreliable information, verifying the PRC has in fact followed through ontheir expressed intention to participate in an activity.

FIG. 5 is a more detailed description of the events that occur in stepS250. In particular, in step S251, the local interaction device 1notifies the PRC that an invitation was generated. Subsequently, theprocess includes two parallel series of steps. For the first series, instep S252 the device awaits an indication that the PRC is ready to takedelivery of the invitation (optional). Subsequently, the processproceeds to step S257, where the device receives an indication that thePRC is ready to take delivery (for example by the PRC's detectedproximity to the device or by the PRC's pressing of the oversized “YES”button). The process then proceeds to step S240 (discussed previously).

In the second series of steps, the device repeats steps S251 and 252 atprescribed intervals (programmable) for a predetermined duration (e.g.,every 10 minutes for 60 minutes). The process then proceeds to step S253where the device does not receive an indication that the PRC is ready totake the delivery. Then, in step S255 the prescribed duration periodexpires and the device determines that a “no response” was entered instep S256, prior to the process proceeding to step S260. In this way,the process provides a failsafe mechanism for providing a response tothe device even if the PRC chooses not to interact with the localinteraction device 1.

FIG. 6 describes in more detail the substeps performed in step S260. Instep S261, the device determines whether the PRC entered a tactileresponse of “YES”, “NO”, request for more information, or offered noresponse. Subsequently, the process proceeds to step S262, where thedevice determines automated actions to take that correspond to thereceived response. In this example, if the PRC enters “YES”, then thedevice sends an e-mail (or other automated response, such as a voicemail indication) to the cooperating entities, in this case a van serviceoperated at the senior center. If the response is “NO”, then the datagets analyzed and compared to predetermined thresholds which will bediscussed with regard to FIG. 7. If no response is received, then theprocess repeats the notification at 10 minute intervals (or otherpredetermined intervals) and if still no response is received after apredetermined number of intervals, then an automated message is sent toa primary Caregiver, in this case perhaps an e-mail to a daughter who isat her office during working hours. If the PRC requests additionalinformation, then the device will provide a reiteration of the originalannouncement as well as a presentation of other information. In oneexample, this other information may contain an audio message from theCaregiver, informing the PRC why this particular activity may beimportant. For example, the Caregiver may include an audio message thatsays “Don't forget that it is Martha's birthday today. You can tell herthis at the bingo game”.

The process then proceeds to step S263 where the device performs anautomated action corresponding to the respective response and then ifthe response is “YES’, the device logs a response, e-mails theconfirmation message and monitors the front door via the door sensor forthe next 60 minutes to confirm that the PRC has in fact left for thebridge club. On the other hand, if the response is “NO”, the device logsa response and analyzes the prior response to determine whether theinactivity exceeds a predetermined level (S264B). In no response isreceived then in step S264C the device repeats the invitation apredetermined number of times and then if necessary contacts the primaryCaregiver. Likewise, if the PRC makes a request for more informationthan in step S264D, the information is provided as discussed above.

FIG. 7 is a flowchart showing how the data received from the localinteraction device 1 is analyzed to determine the status of the PRC. Theprocess begins in step S701, where the data is retrieved at the remoteinterface 3 (or alternatively at the local interaction device 1, or evenat a physician or therapist's office, not shown in this figure). Thedata will include anywhere from one to a larger number of observedactivities. Each activity is optionally given a predetermined weight ofimportance, such that for “essential” functions, an alarm to theCaregiver is provided immediately if the PRC does not respond, orresponds in the negative. On the other hand, if the activity is aroutine, optional activity (bridge game) then monitor 312 (FIG. 1) doesnot generate an alarm unless more than a predetermined number (5 or 6)events are skipped, intentionally or inadvertently.

After collecting the data, the process proceeds to step S703, where thedata on the number of “YES” responses, represented by the variable “X”,is collected. The process then proceeds to step S705 where apredetermined threshold “Y”, is retrieved from memory. Then in stepS707, the monitor compares whether the number of “YES” responses isgreater than the threshold (e.g., X>Y), and the process proceeds to stepS711, if the response to the inquiry in step S707 is affirmative. On theother hand, if the response to the inquiry in step S707 is negative, theprocess proceeds to step S709 where an alert is generated, informing theCaregiver of the status of the PRC. For example, the status indicationmay indicate, event-specific inactivity (failure to participate inbridge games), or general inactivity status, such as failure toparticipate in a variety of different activities.

FIG. 8 is a block diagram of a web-based embodiment of the presentinvention. In this embodiment, the local interaction device includes amodified electronic photo frame 800 (which is a modified version ofphoto frame 1100 in FIG. 11), and a local interface device 803. Thelocal interface device 803 may optionally include all of the features oflocal interaction device 1 (FIG. 3), although there is no need toinclude a display (unless a portable version is used) because themodified electronic photo frame 800 is used as the display. Scheduleevents and monitored activity is performed by the remote interfacedevice 3, although the scheduling data and interaction data isset/collected in a web-server 805. The network 5 interconnects thedevices.

In this embodiment, the modified electronic photo frame 800 operates asa conventional electronic photo frame, displaying photos downloaded fromthe web-server 805. However, when a scheduled event occurs, the modifiedelectronic photo frame 800 displays a graphical (or video, or text)image of the invitation, and also conveys a signal to the localinterface device 803 to produce an audio announcement. The PRC theninteracts with the local interface device 803 to state his or herintentions. The interface data is then sent via the modified electronicphoto frame 800 to the web-server 805 for processing.

Regarding the modifications to the photo frame 800, and peripheralinterface (e.g., USB) that enables the local interface device 803 tocommunication with the processor used in the photo frame 800. Theprocessor in the photo frame 800 is also programmed to convey messagedata between the web-server 805 to the local interface device 803.Furthermore the processor in the electronic photo frame 800 isconfigured to display text messages as well as photos. Also, theprocessor is configured to send messages either a predetermined times,or upon receipt of messages from the web-server 805 or the localinterface device 803.

In this embodiment 805, the scheduling/monitoring software is hosted onthe web-server 805. This allows the Caregiver to access thescheduling/monitor interface at any location.

As an alternative, the local interface device can be built into thephoto-frame, such as, for example, a touchscreen, or as a separatetouchscreen that cooperates with the photo-frame or monitor. Moregenerally, the local interface can be a touchscreen interface thatpresents in a graphical format the “Yes”, “No” and possibly “MoreInformation” areas for selection by the PRC. When the touchscreen is notpresenting an announcement of an event to the PRC, the touchscreen maypresent a graphics, or video image, advertising events that may be ofinterest to the PRC. As a further alternative, the local interfacedevice (or photo-frame) includes a wireless (e.g., Infrared) remotecontrol receiver that acknowledges a first key of a remote control(e.g., up key) as an affirmative response, and a second key on theremote control (e.g., down key) as a negative response. This way, thePRC need not get up from his or her chair or bed to respond to aninvitation to participate in an event. Likewise, this wireless featuremay be included in a watch, pendant, hearing aide, glasses or otherwearable configuration.

The present invention may also be used in a networked environment. Forexample, a coordinator of an assisted care facility or independentliving senior center may have a central remote interface device thatconnects via the network 5 to a plurality of local interaction devices1. This way, the coordinator is provided with a centralizedscheduling/monitoring device that is able to “fuse” information from thedifferent local interaction devices to identify trends, and/or toprovide a coordination service for monitoring/coordinating theactivities of multiple PRCs. In this scenario the coordinator may usethe networked devices to set multiple seatings for meals (e.g., adjustschedules for different subsets of people), thus providing an adaptivescheduling capability.

FIG. 9 illustrates a computer system 1201 upon which an embodiment ofeither the local interaction device 1 or remote interface 3 may beimplemented. The computer system 1201 includes a bus 1202 or othercommunication mechanism for communicating information, and a processor1203 coupled with the bus 1202 for processing the information. Thecomputer system 1201 also includes a main memory 1204, such as a randomaccess memory (RAM) or other dynamic storage device (e.g., dynamic RAM(DRAM), static RAM (SRAM), and synchronous DRAM (SDRAM)), coupled to thebus 1202 for storing information and instructions to be executed byprocessor 1203. In addition, the main memory 1204 may be used forstoring temporary variables or other intermediate information during theexecution of instructions by the processor 1203. The computer system1201 further includes a read only memory (ROM) 1205 or other staticstorage device (e.g., programmable ROM (PROM), erasable PROM (EPROM),and electrically erasable PROM (EEPROM)) coupled to the bus 1202 forstoring static information and instructions for the processor 1203.

The computer system 1201 also includes a disk controller 1206 coupled tothe bus 1202 to control one or more storage devices for storinginformation and instructions, such as a magnetic hard disk 1207, and aremovable media drive 1208 (e.g., floppy disk drive, read-only compactdisc drive, read/write compact disc drive, compact disc jukebox, tapedrive, and removable magneto-optical drive). The storage devices may beadded to the computer system 1201 using an appropriate device interface(e.g., small computer system interface (SCSI), integrated deviceelectronics (IDE), enhanced-IDE (E-IDE), direct memory access (DMA), orultra-DMA).

The computer system 1201 may also include special purpose logic devices(e.g., application specific integrated circuits (ASICs)) or configurablelogic devices (e.g., simple programmable logic devices (SPLDs), complexprogrammable logic devices (CPLDs), and field programmable gate arrays(FPGAs)).

The computer system 1201 may also include a display controller 1209coupled to the bus 1202 to control a display 1210, such as a cathode raytube (CRT), for displaying information to a computer user. The computersystem includes input devices, such as a keyboard 1211 and a pointingdevice 1212, for interacting with a computer user and providinginformation to the processor 1203. The pointing device 1212, forexample, may be a mouse, a trackball, or a pointing stick forcommunicating direction information and command selections to theprocessor 1203 and for controlling cursor movement on the display 1210.In addition, a printer may provide printed listings of data storedand/or generated by the computer system 1201.

The computer system 1201 performs a portion or all of the processingsteps of the invention in response to the processor 1203 executing oneor more sequences of one or more instructions contained in a memory,such as the main memory 1204. Such instructions may be read into themain memory 1204 from another computer readable medium, such as a harddisk 1207 or a removable media drive 1208. One or more processors in amulti-processing arrangement may also be employed to execute thesequences of instructions contained in main memory 1204. In alternativeembodiments, hard-wired circuitry may be used in place of or incombination with software instructions. Thus, embodiments are notlimited to any specific combination of hardware circuitry and software.

As stated above, the computer system 1201 includes at least one computerreadable medium or memory for holding instructions programmed accordingto the teachings of the invention and for containing data structures,tables, records, or other data described herein. Examples of computerreadable media are compact discs, hard disks, floppy disks, tape,magneto-optical disks, PROMs (EPROM, EEPROM, flash EPROM), DRAM, SRAM,SDRAM, or any other magnetic medium, compact discs (e.g., CD-ROM), orany other optical medium, punch cards, paper tape, or other physicalmedium with patterns of holes, a carrier wave (described below), or anyother medium from which a computer can read.

Stored on any one or on a combination of computer readable media, thepresent invention includes software for controlling the computer system1201, for driving a device or devices for implementing the invention,and for enabling the computer system 1201 to interact with a human user(e.g., print production personnel). Such software may include, but isnot limited to, device drivers, operating systems, development tools,and applications software. Such computer readable media further includesthe computer program product of the present invention for performing allor a portion (if processing is distributed) of the processing performedin implementing the invention.

The computer code devices of the present invention may be anyinterpretable or executable code mechanism, including but not limited toscripts, interpretable programs, dynamic link libraries (DLLs), Javaclasses, and complete executable programs. Moreover, parts of theprocessing of the present invention may be distributed for betterperformance, reliability, and/or cost.

The term “computer readable medium” as used herein refers to any mediumthat participates in providing instructions to the processor 1203 forexecution. A computer readable medium may take many forms, including butnot limited to, non-volatile media, volatile media, and transmissionmedia. Non-volatile media includes, for example, optical, magneticdisks, and magneto-optical disks, such as the hard disk 1207 or theremovable media drive 1208. Volatile media includes dynamic memory, suchas the main memory 1204. Transmission media includes coaxial cables,copper wire and fiber optics, including the wires that make up the bus1202. Transmission media also may also take the form of acoustic orlight waves, such as those generated during radio wave and infrared datacommunications.

Various forms of computer readable media may be involved in carrying outone or more sequences of one or more instructions to processor 1203 forexecution. For example, the instructions may initially be carried on amagnetic disk of a remote computer. The remote computer can load theinstructions for implementing all or a portion of the present inventionremotely into a dynamic memory and send the instructions over atelephone line using a modem. A modem local to the computer system 1201may receive the data on the telephone line and use an infraredtransmitter to convert the data to an infrared signal. An infrareddetector coupled to the bus 1202 can receive the data carried in theinfrared signal and place the data on the bus 1202. The bus 1202 carriesthe data to the main memory 1204, from which the processor 1203retrieves and executes the instructions. The instructions received bythe main memory 1204 may optionally be stored on storage device 1207 or1208 either before or after execution by processor 1203.

The computer system 1201 also includes a communication interface 1213coupled to the bus 1202. The communication interface 1213 provides atwo-way data communication coupling to a network link 1214 that isconnected to, for example, a local area network (LAN) 1215, or toanother communications network 1216 such as the Internet. For example,the communication interface 1213 may be a network interface card toattach to any packet switched LAN. As another example, the communicationinterface 1213 may be an asymmetrical digital subscriber line (ADSL)card, an integrated services digital network (ISDN) card or a modem toprovide a data communication connection to a corresponding type ofcommunications line. Wireless links may also be implemented. In any suchimplementation, the communication interface 1213 sends and receiveselectrical, electromagnetic or optical signals that carry digital datastreams representing various types of information.

The network link 1214 typically provides data communication through oneor more networks to other data devices. For example, the network link1214 may provide a connection to another computer through a localnetwork 1215 (e.g., a LAN) or through equipment operated by a serviceprovider, which provides communication services through a communicationsnetwork 1216. The local network 1214 and the communications network 1216use, for example, electrical, electromagnetic, or optical signals thatcarry digital data streams, and the associated physical layer (e.g., CAT5 cable, coaxial cable, optical fiber, etc). The signals through thevarious networks and the signals on the network link 1214 and throughthe communication interface 1213, which carry the digital data to andfrom the computer system 1201 maybe implemented in baseband signals, orcarrier wave based signals. The baseband signals convey the digital dataas unmodulated electrical pulses that are descriptive of a stream ofdigital data bits, where the term “bits” is to be construed broadly tomean symbol, where each symbol conveys at least one or more informationbits. The digital data may also be used to modulate a carrier wave, suchas with amplitude, phase and/or frequency shift keyed signals that arepropagated over a conductive media, or transmitted as electromagneticwaves through a propagation medium. Thus, the digital data may be sentas unmodulated baseband data through a “wired” communication channeland/or sent within a predetermined frequency band, different thanbaseband, by modulating a carrier wave. The computer system 1201 cantransmit and receive data, including program code, through thenetwork(s) 1215 and 1216, the network link 1214 and the communicationinterface 1213. Moreover, the network link 1214 may provide a connectionthrough a LAN 1215 to a mobile device 1217 such as a personal digitalassistant (PDA) laptop computer, or cellular telephone.

The invention may also be implemented by the preparation of applicationspecific integrated circuits or by interconnecting an appropriatenetwork of conventional component circuits, as will be readily apparentto those skilled in the art.

Obviously, numerous modifications and variations of the presentinvention are possible in light of the above teaching. It is thereforeto be understood that within the scope of the appended claims, theinvention may be practiced otherwise than as specifically describedherein.

1. A computer based system for remotely monitoring responses of a personto invitations to participate in activities that include an event,comprising: a local interactive device configured to invite the personto participate in an upcoming event and solicit feedback from the personregarding an intent of the person to participate in the event, saidlocal interactive device including an input/output device configured toreceive a message containing data regarding the event from a remoteinterface device via a communication link, an event reminder mechanismconfigured to generate an announcement signal at a predetermined timenot later than said time for the event, an announcement mechanismconfigured to present an invitation query in audio, and visual form tothe person regarding an intention of the person to participate in theevent, a local interaction device having a first tactile actuator thatwhen actuated only once by the person generates an affirmative responseto the invitation and a second tactile actuator that when actuated onlyonce by the person generates a negative response to the invitation, saidlocal interaction device configured to convert an expression of interestby said person into a digital representation of said interest, and amemory configured to store the digital representation of said expressionof interest, wherein said local interaction device is not either aconventional keyboard or a mouse for a personal computer, said localinteraction device being a self-contained portable device with a keypadhaving not more than three keys, said first tactile actuator includes afirst key of said keypad, and said second tactile actuator includes asecond key of said keypad, said input/output device is configured tosend data corresponding to said expression of interest to said remoteinterface device via said communication link, and said remote interfacedevice including a monitoring mechanism, said monitoring mechanismconfigured to produce cumulative data by combining said data regardingsaid expression of interest with other data received regardingexpressions of interest for participating in different upcoming events,compare said cumulative data against a predetermined threshold, andgenerate an alarm when said cumulative data drops below saidpredetermined threshold, said first tactile actuator having a labelindicative of an affirmative response, and said second tactile actuatorhaving a label indicative of a negative response.
 2. The system of claim1, wherein said self-contained portable unit being at least one of aPDA, cell phone, hearing aide, watch, and bi-focal frame.
 3. The systemof claim 1, wherein said self-contained portable unit being a pendant.